1. Field of the Invention
The present invention relates generally to computer-based methods and apparatuses, including computer program products, for alternate viewpoint rendering.
2. Background
Rendering is often used to refer to a process of taking a model (for example, a three-dimensional (3D) model or an image-based model) and producing from an abstract set of data (e.g., lighting information, texture information, 3D object information, prospective and parallax information) a visual image that is a representation of that data. Rendering is often used to render alternate (or different) viewpoints of an original image. For example, an original image can be rendered into two images that simulate the left and right images that would be perceived by a person's eyes (a stereo pair). Advantageously this process can be used to create synthetic stereopsis (or artificially triggered stereo depth perception).
For a geometric system, an image (or movie) can be rendered using models generated for characters in the image (e.g., computer graphics (CG) models). In some examples, actual physical models can be made (e.g., out of plaster). The models do not need any surface or texture, etc., because the image or movie can be projected onto the model, which provides the proper context (e.g., the blank face of the model will take on the texture and context projected onto it). The projection can also be done virtually inside a CG model system to create the geometry. The models can be moved to approximate the position of the characters on screen. The scene and/or characters are projected onto the model(s), and alternate viewpoints can be generated from different vantage points of the models.
For an image-based approach, conceptually the process is similar to that for a geometric system, but the geometry is not actually built. For example, the geometry in a geometric system is used to show where the light is traveling to and stopping. For a face, light that hits the nose stops at a certain distance, and light that hits the cheek stops at a distance further into the scene than the light impinging upon the nose. Therefore, in the geometric system, geometry is built that interrupts the projected light and indicates where the light lands. In an image-based approach, depth information (such as depth mattes) can be generated without building the actual geometry. The depth information is then used to render alternate viewpoints of the original image.
In the field of motion pictures, it is often cumbersome to shoot a film with two cameras for stereo presentation. Further, there is a large catalog of existing films and television that were created from only one viewpoint. Stereo conversion allows filmmakers to shoot with one camera and achieve the stereo effect as a post-process. However, rendering techniques often suffer from both delays in processing speed and in the accuracy of which the alternate viewpoint images (the rendered images) are generated. For example, if it takes too long to generate stereo pairs from source images, it becomes impracticable to render sequences of original images (e.g., movies). Further, while the images can be filtered to speed up the process, the alternate viewpoint images may suffer from excessive distortion. This distortion can compromise the viewing pleasure of the rendered sequences, which results in a poor viewer experience.